Field
The present disclosure relates generally to augmented reality (AR) devices, e.g., AR eyeglasses, having optical see-through head mounted displays (HMD) and user interaction capability, and more particularly, to conducting online visual searches in real world using such AR devices. AR is a technology in which a user's view of the real world is enhanced with additional information generated from a computer model. The enhancements may include labels, 3D rendered models, or shading and illumination changes. AR allows a user to work with and examine the physical real world, while receiving additional information about the objects in it.
Background
AR devices typically include an optical see-through HMD and one or more user input mechanisms that allow users to simultaneously see and interact with their surroundings while interacting with applications, such as e-mail and media players. User input mechanisms may include one or more of gesture recognition technology, eye tracking technology, and other similar mechanisms.
In AR devices, also referred to herein as HMD systems, virtual objects displayed in the HMD screen augment real world scenes viewed by a user through the HMD. For example, a person in the field of view of a user may be augmented with her name, an artwork may be augmented with descriptive information, and a book may be augmented with its price and so on.
With such an HMD system, a user can perform a visual search to obtain more information about a specific object visible in a real-world scene, i.e., within the field of view of the user. During the visual search, the HMD system generally takes a snapshot of the scene in the field of view, conducts a search using an online visual search engine (e.g. Google Goggles), retrieves the results, and displays the results. The results are displayed on the HMD screen to provide an augmented reality scene to the user that includes the real-world scene, with the object just searched visible through the HMD, and the augmented reality search result.
A problem with this visual search approach is that other objects may be present in the real-world scene resulting in inaccurate search results and redundant or extraneous data transfer to the online visual search engine. To work around this issue, a user may have to move closer to the object of interest so that the object of interest approximately covers her entire field of view. Although this may work for visual searching with a Smartphone, wherein a user zooms in on the object to be searched and takes a picture, this approach is not desirable for optical see-through HMD systems in that its requires the user to move and thus may not provide a good user experience.